Measuring Advertising Quality on Television Deriving Meaningful Metrics from Audience Retention Data

December 2009 JOURNAL OF ADVERTISING RESEARCH 419 measure their success through user-response metrics such as click-through rate (CTR), conversion rate (Richardson, Dominowski, and Ragno, 2007), and bounce rate (Sculley, Malkin, Basu, and Bayardo, 2009), Google has been exploring how to use STB measurement to design equivalent measures for TV. Past attempts to provide quality scores for TV ads have typically relied on smaller constructed panels and focused on programming with very large audiences. For example, for the 2009 Super Bowl, Nielsen published likeability and recall scores for the top ads (Nielsen Inc., 2009). The scores were computed using 11,466 surveys, and they reported on the five best-liked ads and the five most-recalled ads. In this article, we define a rigorous measure of audience retention for TV ads that can be used to predict future audience response for a much larger range of ads. The primary challenge in designing such a measure is that many factors appear to impact STB tuning during ads, making it difficult to isolate the effect of the specific ad itself on the probability that a STB will tune away. We propose several ways of modeling such a probability. To the best of our knowledge, this is the first to attempt INTRODUCTION In recent years, there has been an explosion of interest in collecting and analyzing television settop box (STB) data (also called “return-path” data) (Bachman 2009). As U.S. television moves from analog to digital signals, digital STBs increasingly are common in American homes. Where these are attached to some sort of return path (as is the case in many homes subscribing to cable or satellite TV services), these data can be aggregated and licensed to companies wishing to measure television viewership. Advances in distributed computing make it feasible to analyze these data on a massive scale. Whereas previous television measurement relied on panels consisting of thousands of households, data can now be collected and analyzed for millions of households. This holds the promise of providing accurate measurement for much (and perhaps all) of the niche TV content that eludes current panel-based methods in many countries. In addition to using these data for raw audience measurement, it should be possible to make more qualitative judgments about the content—and specifically the advertising—on television. In much the same way that online advertisers frequently Measuring Advertising Quality on Television Deriving Meaningful Metrics from Audience Retention Data